Method and apparatus for capturing video data

ABSTRACT

A method for generating video data using an imaging device is provided. The method includes capturing, by a first sensor of the imaging device, a first set of sub-sampled monochrome frames, capturing, by a second sensor of the imaging device, a second set of sub-sampled color frames, reconstructing one or more high resolution monochrome frames using the first set of sub-sampled monochrome frames, and generating one or more high speed high resolution color frames based on the second set of sub-sampled color frames and the reconstructed one or more high resolution monochrome frames.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. §119(a) to an Indian provisional patent application filed on May 17,2017, in the Indian Patent Office and assigned Serial number201741017337, and to an Indian patent application filed on Apr. 16,2018, in the Indian Patent Office and assigned Serial number201741017337, the disclosure of each of which is incorporated byreference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to capturing video. More particularly, thedisclosure relates to capturing high speed high quality video using dualsensors.

2. Description of the Related Art

High speed imaging refers to methods or techniques for taking images orvideos of very fast events.

Images are captured in a shorter exposure window and lower exposure timecausing light-starved conditions due to the prevalence of a high framerate. Images are affected due to a lower sensitivity resulting from thelight-starved condition.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providea method and apparatus for capturing high speed high quality video usingdual sensors.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, a method for generatingvideo data using an imaging device is provided. The method includescapturing, by a first sensor of the imaging device, a first set ofsub-sampled monochrome frames, capturing, by a second sensor of theimaging device, a second set of sub-sampled color frames, reconstructingone or more high resolution monochrome frames using the first set ofsub-sampled monochrome frames, and generating one or more high speedhigh resolution color frames based on the second set of sub-sampledcolor frames and the reconstructed one or more high resolutionmonochrome frames.

In an embodiment, the one or more high resolution monochrome frames areused as a guide for color propagation in the one or more high speed highresolution color frames.

In an embodiment, the one or more high resolution monochrome frames isreconstructed using a flexible sub-sampled readout (FSR) mechanism.

In an embodiment, the second set of sub-sampled color frames is moresparsely sub-sampled than the first set of sub-sampled monochromeframes.

In an embodiment, the method includes the first sensor is a monochromesensor and the second sensor is a Bayer sensor.

In an embodiment, reconstructing the one or more high resolutionmonochrome frames from the first set of sub-sampled monochrome framesfurther includes capturing a plurality of parity fields from the firstsensor of the imaging device using a multi parity FSR mechanism, whereinthe imaging device is in an FSR mode. The method further includesreconstructing the one or more high resolution monochrome frames fromthe plurality of parity fields using an FSR reconstruction, wherein theFSR reconstruction utilizes one of a pre-image signal processor (ISP)FSR reconstruction and a post-ISP FSR reconstruction based on abandwidth capacity of an ISP of the imaging device.

In accordance with another aspect of the disclosure, an imaging devicefor generating video data, is provided. The imaging device includes afirst sensor configured to capture a first set of sub-sampled monochromeframes, a second sensor configured to capture a second set ofsub-sampled color frames, and a processor coupled to the first sensorand the second sensor, the processor being configured to reconstruct oneor more high resolution monochrome frames using the first set ofsub-sampled monochrome frames, and generate one or more high speed highresolution color frames based on the second set of sub-sampled colorframes and the reconstructed one or more high resolution monochromeframes.

Other aspects advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 illustrates various hardware components of an electronic devicewith a dual sensor for high speed imaging, according to an embodiment ofthe disclosure;

FIG. 2 is a flow chart illustrating a method for Bayer reconstructionfrom sparse sampled data with a dual sensor, according to an embodimentof the disclosure;

FIG. 3 illustrates a monochrome reconstruction and color propagationresults, according to an embodiment of the disclosure;

FIG. 4 is a high level overview of a system for capturing a high speedhigh quality video using a dual sensor, according to an embodiment ofthe disclosure;

FIG. 5 illustrates a monochrome and color reconstruction, according toan embodiment of the disclosure;

FIG. 6 is an example scenario illustrating high speed high quality videorecording, according to an embodiment of the disclosure;

FIG. 7 is an example scenario illustrating capturing high resolution lowlight images, according to an embodiment of the disclosure;

FIG. 8 is an example scenario illustrating high dynamic range (HDR)video recording at a high frame rate, according to an embodiment of thedisclosure;

FIG. 9 is an example scenario illustrating high quality motion de-blur,according to an embodiment of the disclosure; and

FIG. 10 is an example scenario illustrating differential frame ratevideo recording, according to an embodiment of the disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION OF INVENTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

It may be advantageous to set forth definitions of certain words andphrases used throughout this document. The terms “include” and“comprise,” as well as derivatives thereof, mean inclusion withoutlimitation. The term “or,” is inclusive, meaning and/or. The phrases“associated with” and “associated therewith,” as well as derivativesthereof, may mean to include, be included within, interconnect with,contain, be contained within, connect to or with, couple to or with, becommunicable with, cooperate with, interleave, juxtapose, be proximateto, be bound to or with, have, have a property of, or the like.

Moreover, various functions described below can be implemented orsupported by one or more computer programs, each of which is formed fromcomputer readable program code and embodied in a computer readablemedium. The terms “application” and “program” refer to one or morecomputer programs, software components, sets of instructions,procedures, functions, objects, classes, instances, related data, or aportion thereof adapted for implementation in a suitable computerreadable program code. The phrase “computer readable program code”includes any type of computer code, including source code, object code,and executable code. The phrase “computer readable medium” includes anytype of medium capable of being accessed by a computer, such as readonly memory (ROM), random access memory (RAM), a hard disk drive, acompact disc (CD), a digital video disc (DVD), or any other type ofmemory. A “non-transitory” computer readable medium excludes wired,wireless, optical, or other communication links that transporttransitory electrical or other signals. A non-transitory computerreadable medium includes media where data can be permanently stored andmedia where data can be stored and later overwritten, such as arewritable optical disc or an erasable memory device.

Definitions for certain words and phrases are provided throughout thisdocument, and those of ordinary skill in the art should understand thatin many, if not most instances, such definitions apply to prior, as wellas future uses of such defined words and phrases.

Currently, maximum achievable frame-rate for a motion capturing imagingdevice is limited by pixel readout rate of an imaging sensor of theimaging device. The imaging sensors used currently may allow either aslow frame-rate at full resolution (for example, 30 frames per second(fps) at ultra high definition (UHD) resolution or a fast frame-rate atlow resolution (for example, 120 fps at full high definition (FHD)resolution). Higher frame-rates can be achieved using pixel readoutmodes such as sub-sampling, binning or the like but usage of the pixelreadout modes sacrifices spatial for temporal resolution within a fixedbandwidth.

While higher frame rate can be achieved at a lower resolution, a fieldof view with respect to the captured image is affected. Also, dynamicchanging of a region of interest is also difficult to achieve at a lowerresolution. Other limitations of existing methods of high speed imagecapture include poor multiple camera synchronization of dual views. Highspeed imaging requires higher international standards organization (ISO)settings. Higher ISO results in noisier or grainy images.

Using dual color-monochrome sensors for high speed imaging also resultsin differing image characteristics. For example, the color sensor canhave greater resolution loss compared to the monochrome sensor. Themonochrome sensor captures more light giving better brightness, lessnoise and better resolution.

Thus, there is a need for a dual sensor mechanism that addresses theshortcomings of conventional techniques of high speed imaging. The aboveinformation is presented as background information only to help thereader to understand the disclosure. Applicants have made nodetermination and make no assertion as to whether any of the above mightbe applicable as prior art with regard to the present application.

FIG. 1 illustrates various hardware components of an electronic devicewith a dual sensor for high speed imaging, according to an embodiment ofthe disclosure.

Referring to FIG. 1, an electronic device 100 includes a monochromesensor 102, a Bayer sensor 104, a reconstruction unit 106, a display108, an image signal processor (ISP) 110 and a memory 112. Thereconstruction unit 106 and the ISP 110 may be implemented as onehardware processor. The electronic device 100 can include aninput/output (I/O) interface (not shown) that can be but is not limitedto a web interface, a graphical user interface for the display 108, aninterface for the Bayer sensor 104 and the monochrome sensor 102 and thelike. The I/O interface can communicate with other devices or a systemthrough a communication network. The communication network can include adata network such as, but not limited to, an Internet, local areanetwork (LAN), wide area network (WAN), metropolitan area network (MAN)etc. In another embodiment, the communication network can include awireless network, such as, but not limited to, a cellular network andmay employ various technologies including enhanced data rates for globalevolution (EDGE), general packet radio service (GPRS), global system formobile communications (GSM), Internet protocol multimedia subsystem(IMS), universal mobile telecommunications system (UMTS) etc.Accordingly, the electronic device 100 is equipped with communicationcomponents facilitating communications over the communication network.In some embodiments, the electronic device 100 can be part of anInternet of things (IoT) network.

The monochrome sensor 102 may capture a set of sub-sampled monochromeframes of a subject. The Bayer sensor 104 may capture a set ofsub-sampled Bayer frames of the subject using multiple parity flexiblesub-sampled readout (FSR) to generate a plurality of parity fields. Theparity fields pertaining to the Bayer data are at a lower resolutionrelative to the monochrome data. The monochrome data and the Bayer datacan be stored in the memory 112. In another embodiment, the memory 112can be a circular memory. Whenever an image is to be captured by a userusing the high speed imaging method according to an embodiment of thedisclosure, the user can use any gesture such as a tap on the I/Ointerface. The gesture may be detected as a trigger event that initiatesthe reconstruction mechanism. The reconstruction unit 106 is configuredto fetch the parity fields pertaining to the monochrome data and theBayer data that lie within predefined time units around the triggerevent stored in the memory 112. Upon detecting the trigger event, theBayer data may be stored in the memory 112 and frozen in the memory 112to prevent overwriting.

In another embodiment, the trigger event can also be detectedautomatically by analyzing motion in captured video frames. Automatictriggering of the event starts the capturing of high frame rate datafrom the monochrome sensor 102.

In another embodiment, the reconstruction unit 106 can performde-mosaicing on the parity fields to generate high resolution monochromeframes and red green, and blue (RGB) frames, respectively.

The reconstruction unit 106 utilizes a pre-ISP FSR reconstruction and apost-ISP FSR reconstruction based on a bandwidth capacity of the ISP110. The reconstruction unit 106 uses the sub-sampled monochrome framesas a guide for propagating color. The reconstruction unit 106 mayreconstruct high resolution monochrome frames from the sub-sampledmonochrome data and high speed high resolution color frames from theBayer data and the high resolution monochrome frames.

In an implementation, the Bayer data is more sparsely sub-sampled thanthe monochrome data.

The ISP 110 can be, but is not limited to, a central processing unit(CPU), a microprocessor, or a microcontroller. The ISP 110 may becoupled to the memory 112, the monochrome sensor 102 and the Bayersensor 104, and the display 108. The ISP 110 may execute sets ofinstructions stored on the memory 112.

The memory 112 includes storage locations to be addressable through theISP 110. The memory 112 is not limited to a volatile memory and/or anon-volatile memory. Further, the memory 112 can include one or morecomputer-readable storage media. The memory 112 can include non-volatilestorage elements. For example, non-volatile storage elements can includemagnetic hard discs, optical discs, floppy discs, flash memories, orforms of electrically programmable memories (EPROM) or electricallyerasable and programmable (EEPROM) memories. In an embodiment, thememory 112 can be a circular memory.

FIG. 2 is a flow chart illustrating a method for Bayer reconstructionfrom sparse sampled data with the dual sensor, according to anembodiment of the disclosure.

Referring to FIG. 2, in operation 202, the user triggers the imagereconstruction through a triggering event such as a user input such as agesture input, pushing a button, or an auto trigger input based onmotion detection between image frames on the I/O interface of theelectronic device 100. In operations 204 and 226, the monochrome sensor102 and the Bayer sensor 104 initiate their respective sub-samplingmechanisms. In the first sensor path, as shown in FIG. 2, thereconstruction unit 106 receives the sub-sampled monochrome data fromthe monochrome sensor 102. The first monochrome frame is captured infull, in operation 206. The first frame is used as a reference tocalibrate the monochrome sensor to enhance focus and sensitivity, inoperation 208. In another embodiment, the first monochrome frame iscaptured without any subsampling and is used to calibrate the monochromesensor 102.

Subsequent frames are captured through an FSR mechanism. Thereconstruction unit 106 captures the plurality of parity fieldspertaining to the monochrome sensor 102 of the electronic device 100.The plurality of parity fields provide spatial and temporal sampling ofthe full frame monochrome data. The plurality of parity fields capturethe full frame monochrome data based on a multi parity FSR mechanismthat utilizes a zigzag sampling pattern or the like. Further, thereconstruction unit 106 reconstructs the high speed high resolutionmonochrome frames from the plurality of parity fields using the FSRreconstruction, wherein the FSR reconstruction utilizes pre-ISP FSRreconstruction or post-ISP FSR reconstruction based on a bandwidthcapacity of the ISP 110 of the electronic device 100.

In the multi parity FSR, each of the plurality of parity fieldsobtained, that captures the full frame data through sampling, isassigned a parity number iteratively, as shown in operations 212, 214,and 216. Further, each of the plurality of parity fields has a scaledframe size and a scaled frame rate that is obtained by using the zigzagsampling pattern or the like. The frame size of the parity fields isreduced by a scaling factor and the frame rate is enhanced by thescaling factor. The scaling factor is equal to a number of distinctparity patterns (each identified by a unique parity number) used by thezigzag sampling pattern or similar sampling patterns used. For example,if the maximum current imaging sensor read out frame rate is N, then themulti parity FSR enhances the frame rate to ‘scaling factor times N”(scaling factor N). In FIG. 2, as an embodiment, a four parity field isconsidered. The four parity field (parity pattern) of the multi parityFSR has four distinct parity fields such as parity 1, parity 2, parity 3and parity 4, which are repeated in sequence for the plurality of parityfields generated. Thus, the 4 parity readout increases the frame rate byfactor 4 and reduces the frame size or resolution by four for eachparity field, as per operations 220, 222, and 224 of FIG. 2. Similarly,a selection of an eight field parity can further increase the frame rateand reduce the frame size by scaling factor 8.

The plurality of parity fields capture the full frame monochrome data ofthe monochrome sensor 102 to provide sampled full frame monochrome data.The plurality of parity fields are stored in the memory 112. The memory112 may retain the plurality of parity fields for a predefined timeunits for example (2N seconds) based on a first-in-first-out (FIFO)mechanism.

Once the parity fields are generated, the reconstruction unit 106 can beconfigured to reconstruct the high speed high resolution monochromeframes by applying the FSR reconstruction mechanism on the plurality ofparity fields.

In an embodiment, the FSR reconstruction mechanism may utilize thepre-ISP FSR reconstruction when the ISP 110 has a low bandwidthcapacity, where the FSR reconstruction for generating the high speedhigh resolution monochrome frames is performed before performing ISPprocessing.

In an embodiment, the FSR reconstruction is based on the post-ISP FSRreconstruction when the ISP 110 supports a higher bandwidth, where theFSR reconstruction generating the high speed high resolution monochromeframes is performed after the ISP processing.

In the second sensor path, as shown in FIG. 2, of operation 226, thefirst Bayer frame is captured and chromaticity is estimated in eachBayer block (operations 238 and 240). Subsequent frames are captured infull, in operation 228. The reconstruction unit 106 performsde-mosaicing on the Bayer data comprising captured Bayer frames togenerate RGB frames, in operation 230. The de-mosaiced Bayer data can bepreviewed on a screen of the display 108. The de-mosaiced Bayer data maybe converted to grayscale, in operation 232. In operation 234, the Bayerdata is projected onto the homographic matrix determined for themonochrome data. Subsequently, the high resolution monochrome frames maybe utilized as a guide for color propagation. In operation 236, thereconstruction unit 106 reconstructs high resolution image frames of thesubject based on the high resolution monochrome frames and the Bayerdata. The reconstructed frames are displayed on the display 108.

FIG. 3 illustrates a monochrome reconstruction and color propagationresults, according to an embodiment of the disclosure.

Referring to FIG. 3, the monochrome sensor 102 captures sub-sampledmonochrome frames that are reconstructed as per operations 302 and 304similar to operations of 204, 206, 208, 212, 214, 216, 220, 222, and 224in FIG. 2. The high resolution monochrome frames act as a guide to theBayer data to eventually reconstruct high resolution high speed imageframes 306 of the subject.

FIG. 4 is a high level overview of a system for capturing a high speedhigh quality video data using a dual sensor, according to an embodimentof the disclosure.

Referring to FIG. 4, the monochrome sensor 102 captures FHD monochromeframes at 240 fps. Using the FSR mechanism described in connection withFIG. 2, the monochrome frames are reconstructed at 1000 fps with aparity number of four, in operation 402. The monochrome frames are readat a quarter size of the captured frames captured by the monochromesensor 102. The Bayer data can also be read at a quarter size, i.e. at1000 fps. The Bayer data is further de-mosaiced and converted tograyscale. The FHD monochrome frames at 1000 fps are used as a guide forcolor propagation in the Bayer frames, as seen in operation 404. Theresultant reconstructed color frames 406 are of high resolution (at 1000fps color).

FIG. 5 illustrates a monochrome and color reconstruction, according toan embodiment of the disclosure.

Referring to FIG. 5, similar to FIG. 4, the reconstruction unit 106implements flexible read out reconstruction on sparse monochrome framescaptured by the monochrome sensor 102, in operation 504. The highresolution monochrome frames act as a guide for color propagation tode-mosaiced grayscale frames reconstructed from sparse Bayer datacaptured by the Bayer sensor 104. The resultant reconstructed frames 508after color propagation, implemented in operation 506, become highresolution images.

FIG. 6 is an example scenario illustrating high speed high quality videorecording, according to an embodiment of the disclosure.

Referring to FIG. 6, FIG. 6 illustrates fetching of a plurality ofparity fields for the reconstruction of the high frame rate highresolution video using one of the FSR reconstruction mechanism on thedetection of user's event triggering the FSR reconstruction, accordingto an embodiment of the disclosure. FIG. 6 depicts a preview video framesequence on the display 108 displayed at 30 fps. From video framescaptured by the monochrome sensor 102 and the Bayer sensor 104, theplurality of parity fields are generated and stored on the memory 112.

Further, whenever the user intends to capture video of interest at highframe rate high resolution (FSR mode), the user may use a predefinedgesture such as tap on the record icon of a corresponding cameraapplication running in FSR mode. The gesture is detected as a triggerevent that triggers the FSR reconstruction mechanism. Once the triggerevent is detected, the reconstruction unit 106 fetches the plurality ofFHD parity fields currently retained in the memory 112 for a time spanof 2N seconds (−N (first set of parity fields) to +N (second set ofparity fields)=2N seconds), in operation 604. As depicted in FIG. 6, the2N seconds span is distributed around the user event with N secondsbefore and N seconds after the user event. This enables capturing thevideo of interest more precisely by considering video frames adjacent tothese and eliminating any chance of losing a video event of interest 602due to lags between user action and actual capturing. The plurality ofparity fields that do not lie within the time span are discarded due toFIFO action and hence memory space consumed for the plurality of parityfields generated is static even though the FSR mode is enabled in theelectronic device 100 for a longer duration. The electronic device 100,through usage of circular memory, reduces consumption of memory by notretaining the undesired high frame rate parity field data. The pluralityof parity fields generated continuously, are overwritten over old parityfields data. Each frame of the video event of interest 602 is capturedas high resolution monochrome frames and Bayer data and a highresolution video 608 is reconstructed, in operation 606, by using themonochrome frames as a guide for color propagation.

FIG. 7 is an example scenario illustrating capturing high resolution lowlight images, according to an embodiment of the disclosure.

Referring to FIG. 7, the monochrome sensor 102 may capture frames 702 atbetter light due to the absence of a filter array such as an RGB filterarray. The Bayer sensor 104 may capture low quality frames 704 in lowlight. Using the monochrome frames reconstructed using the FSRmechanism, the resultant reconstructed image frames 706 have improvedilluminance.

FIG. 8 is an example scenario illustrating high dynamic range (HDR)video recording at high frame rate, according to an embodiment of thedisclosure.

Referring to FIG. 8, the usage of dual sensors results in varyingexposures. For example, monochrome frames can be captured with a longexposure by the monochrome sensor 102 while the Bayer frames arecaptured at a relatively shorter exposure. In an embodiment of thedisclosure, each of dual sensors adopted in the electronic device 100have different exposure time and sensitivity, respectively, in capturingimages. As shown in FIG. 8, the long exposure frames have greatersensitivity and are used as a guide to reconstruct the Bayer frames. Inoperation 802, the respective image frames are reconstructed based onthe varying sensitivity. The varying sensitivities are blended duringoperation 804 to obtain image frames 806 at high dynamic ranges.

FIG. 9 is an example scenario illustrating high quality motion de-blur,according to an embodiment of the disclosure.

Referring to FIG. 9, motion blur is caused while capturing a movingobject from a single exposure camera. In the example scenario, we applya short exposure chosen with a binary pseudo random sequence. Eachparity sub-sampled data is exposed to a short exposure chosen with abinary pseudo random sequence. This coded exposure preserves highfrequency spatial details in the blurred image and deconvolution toapply de-blur becomes easy.

FIG. 10 is an example scenario illustrating differential frame ratevideo recording, according to an embodiment of the disclosure.

Referring to FIG. 10, FIG. 10 illustrates selective high frame ratecapture for only a region of interest (ROI) in the plurality of videoframes 1002 being captured, according to an embodiment of thedisclosure. In operation 1006, the reconstruction unit 106 captures andreconstructs the high frame rate high resolution video for a ROI of eachvideo frame providing 1000 fps, while in operation 1004 the remainingvideo frames are captured at a normal or lower frame rate of 30 fps. Themode generates the plurality of parity fields for only the ROI of themonochrome data and the Bayer data. Since only a portion of the video isrecorded at a higher frame rate, the bandwidth limitation required inthe differential mode is reduced.

The embodiments disclosed herein can be implemented through at least onesoftware program running on at least one hardware device and performingnetwork management functions to control the elements. The elements shownin FIGS. 1-10 include blocks which can be at least one of a hardwaredevice, or a combination of hardware device and software module.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. A method for generating video data using an imaging device, the method comprising: capturing, by a first sensor of the imaging device, a first set of sub-sampled monochrome frames; capturing, by a second sensor of the imaging device, a second set of sub-sampled color frames; reconstructing one or more high resolution monochrome frames using the first set of sub-sampled monochrome frames and a flexible sub-sampled readout (FSR) mechanism; and generating one or more high speed high resolution color frames based on the second set of sub-sampled color frames and the reconstructed one or more high resolution monochrome frames.
 2. The method of claim 1, wherein the first sensor and the second sensor have different exposure times and sensitivities, in capturing images.
 3. The method of claim 1, wherein the one or more high resolution monochrome frames are used as a guide for color propagation in the one or more high speed high resolution color frames.
 4. The method of claim 1, wherein a detection of a predefined user gesture triggers the FSR mechanism.
 5. The method of claim 1, wherein the second set of sub-sampled color frames is more sparsely sub-sampled than the first set of sub-sampled monochrome frames.
 6. The method of claim 1, wherein the reconstructing is initiated based on a user trigger made on at least one of preview images corresponding to the one or more high speed high resolution color frames.
 7. The method of claim 1, wherein the first sensor comprises a monochrome sensor and the second sensor comprises a Bayer sensor.
 8. The method of claim 7, wherein the monochrome sensor comprises a non-color sensor and the Bayer sensor comprises a color sensor.
 9. The method of claim 1, wherein reconstructing of the one or more high resolution monochrome frames from the first set of sub-sampled monochrome frames comprises: capturing a plurality of parity fields from the first sensor of the imaging device using a multi parity FSR mechanism in an FSR mode; and reconstructing the one or more high resolution monochrome frames from the plurality of parity fields using an FSR reconstruction, and wherein the FSR reconstruction utilizes one of a pre-image signal processor (ISP) FSR reconstruction and a post-ISP FSR reconstruction based on a bandwidth capacity of a processor included in the imaging device.
 10. An imaging device for generating video data, the imaging device comprising: a first sensor configured to capture a first set of sub-sampled monochrome frames; a second sensor configured to capture a second set of sub-sampled color frames; and a processor coupled to the first sensor and the second sensor, the processor being configured to: reconstruct one or more high resolution monochrome frames using the first set of sub-sampled monochrome frames and a flexible sub-sampled readout (FSR) mechanism, and generate one or more high speed high resolution color frames based on the second set of sub-sampled color frames and the reconstructed one or more high resolution monochrome frames.
 11. The imaging device of claim 10, wherein the first sensor and the second sensor have different exposure times and sensitivities, in capturing images.
 12. The imaging device of claim 10, wherein the processor is further configured to use the one or more high resolution monochrome frames as a guide for color propagation for the one or more high speed high resolution color frames.
 13. The imaging device of claim 10, wherein a detection of a predefined user gesture triggers the FSR mechanism.
 14. The imaging device of claim 10, wherein the second set of sub-sampled color frames are more sparsely sub-sampled than the first set of sub-sampled monochrome frames.
 15. The imaging device of claim 10, wherein the reconstructing is initiated based on a user trigger made on at least one of preview images corresponding to the one or more high speed high resolution color frames.
 16. The imaging device of claim 10, wherein the first sensor comprises a monochrome sensor and the second sensor comprises a Bayer sensor.
 17. The imaging device of claim 16, wherein the monochrome sensor comprises a non-color sensor and the Bayer sensor comprises a color sensor.
 18. The imaging device of claim 10, wherein the processor is further configured to reconstruct the one or more high resolution monochrome frames from the first set of sub-sampled monochrome frames by: capturing a plurality of parity fields from the first sensor of the imaging device using a multi parity FSR mechanism in an FSR mode; and reconstructing the one or more high resolution monochrome frames from the plurality of parity fields using an FSR reconstruction, and wherein the FSR reconstruction utilizes one of a pre-image signal processor (ISP) FSR reconstruction and a post-ISP FSR reconstruction based on a bandwidth capacity of the processor.
 19. A non-transitory computer program product comprising a computer readable storage medium having a computer readable program stored therein, the computer readable program, when executed on a computing device, causing the computing device to: capture, with a first sensor, a first set of sub-sampled monochrome frames; capture, with a second sensor, a second set of sub-sampled color frames; reconstruct one or more high resolution monochrome frames using the first set of sub-sampled monochrome frames and a flexible sub-sampled readout (FSR) mechanism; and generate one or more high speed high resolution color frames based on the second set of sub-sampled color frames and the reconstructed one or more high resolution monochrome frames.
 20. The method of claim 1, wherein the reconstructing of the one or more high resolution monochrome frames using the first set of sub-sampled monochrome frames comprises, reconstructing the one or more high resolution monochrome frames using the first set of sub-sampled monochrome frames in response to a triggering event. 